3-dimensional image creating apparatus, 3-dimensional image reproducing apparatus, 3-dimensional image processing apparatus, 3-dimensional image processing program and recording medium recorded with the program

ABSTRACT

An image signal composed of sequential frames is input to a 3-dimensional image creating apparatus, frame by frame. A controller ( 102 ) designates the presence/absence of reduction, the presence/absence of joining and 2D select. An image converter ( 101 ) creates image data in the format designated by the presence/absence of reduction and the presence/absence of joining. A 3D information creator ( 103 ) creates 3D information necessary for displaying the image as a 3-dimensional image by formatting the presence/absence of reduction, the presence/absence of joining and 2D select. A multiplexer ( 104 ) converts image data and 3D information in a predetermined format and outputs them to the outside. In this way, it is possible to make the image data for 3-dimensional display versatile and select an arbitrary viewpoint image efficiently.

This application is a Divisional of application Ser. No. 10/550,710filed on Sep. 26, 2005 now U.S. Pat. No. 7,636,088, which is a NationalPhase of PCT/JP2004/005484 filed on Apr. 16, 2004, and for whichpriority is claimed under 35 U.S.C. §120; and these applications claimpriority of Application No. JP2003-130711 filed in Japan on May 8, 2003and JP2003-112801 filed in Japan on Apr. 17, 2003 under 35 U.S.C. §119;the entire contents of all are hereby incorporated by reference.

TECHNICAL FIELD

The present invention relates to a 3-dimensional image creatingapparatus for adding attribute information to image data when image datafor 3-dimensional display is created and also relates to a 3-dimensionalimage reproducing apparatus for reproducing the data.

The present invention also relates to a 3-dimensional image processingapparatus, a 3-dimensional image processing program and a recordingmedium recorded with the program, which are aimed at, being able to warnthe user about that status when the parallax quantity varies due toenlargement or reduction in size of 3-dimensional images so that thereis a fear that it becomes difficult to show the magnified 3-dimensionalimages to a stereoscopic image or to present a stereoscopic effect, andmaking correction if the stereoscopic effect can be reformed.

BACKGROUND ART

Conventionally, various methods have been proposed which display3-dimensional images. Of these, “binocular methods” using binocularparallax are generally used. Specifically, a stereoscopic view isachieved by providing left and right-eye images having binocularparallax and projecting them separately on the left and right eyes,respectively.

FIG. 16 is a conceptual view for illustrating a “alternating-fieldsystem” as one of the typical binocular methods.

In this alternating-field system, the left-eye image and right-eye imageare interlaced on alternate horizontal lines of one pixel as shown inFIG. 16, so that the left-eye image and right-eye image will be switchedand displayed alternately. The left-eye image and right-eye imagetherefore have half the vertical resolution compared to that in normal2-dimensional display mode. An observer should put on shutter glassesthat open and close in synchronism with the switching period of thedisplay. The shutter used here opens the left-eye side and closes theright-eye side when the left-eye image is displayed and closes theleft-eye side and opens the right-eye side when the right-eye image isdisplayed. With this arrangement, the left-eye image is observed by theleft eye alone while the right-eye image is observed by the right eyealone, to achieve stereoscopic view.

FIG. 17 is a conceptual view for illustrating another typical scheme ofthe binocular methods, namely “parallax barrier system”.

FIG. 17( a) is a view showing the principle of the cause of parallax.FIG. 17( b) is a view showing an image frame displayed in the parallaxbarrier system.

In FIG. 17( a), an image in which the left-eye image and right-eye imageare interlaced on alternate vertical lines of one pixel as shown in FIG.17( b), is displayed on an image display panel 401 while a parallaxbarrier 402 with slits having a slit width smaller than the intervalbetween the pixels for an identical viewpoint is placed in front ofimage display panel 401, whereby the left-eye image is observed by theleft eye 403 alone while the right-eye image is observed by the righteye 404 alone, to achieve stereoscopic view.

Incidentally, there is another method, the “lenticular system” forachieving 3-dimensional display of an image as shown in FIG. 17( b),which is similar to the parallax barrier system. One example of arecording data format used in the lenticular system is disclosed byJapanese Patent Application Laid-open Hei 11-41627.

FIG. 18 is a conceptual view showing one example of a recording dataformat of the lenticular system. A left-eye image 501 as shown in FIG.18( a) and a right-eye image 502 as shown in FIG. 18( b) are eachthinned to half with respect to the horizontal direction, forming andrecording a frame of complex image 503 as shown in FIG. 18( c). Uponreproduction, this complex image 503 is rearranged to form a compositeimage as shown in FIG. 17( b).

Although not limited to 3-dimensional images, Japanese PatentApplication Laid-open No. 2001-337994 discloses a method of storingadditional information for identification of a thumbnail image anddisplaying the additional information laid over the thumbnail image on adisplay device.

In the above way, in the method of achieving stereoscopic view byletting the left and right eyes observe different images, it is possibleto practice comfortable stereoscopic view when the distance ofcounterpart points of the left and right images (will be calledparallax, hereinbelow) falls within a certain fixed range. However, asthe parallax becomes greater, the images for both eyes will not mergeinto a stereoscopic view. The magnitude of the parallax at that point,has been reported by, for example “a tentative plan of guidelines for3-dimensional images” published in 2002 by The Mechanical Social SystemsFoundation.

Japanese Patent Application Laid-open 2000-78615 and Japanese PatentApplication Laid-open Hei 10-221775 disclose methods of achieving easydisplay of a 3-dimensional image, when its binocular images are hard tomerge into a stereoscopic view due to the magnitude of the aboveparallax, by shifting the displayed positions of left and right imageson a stereoscopic display so as to adjust the parallax.

As stated above, in conventional 3-dimensional display systems,recording of data is done in a fixed recording data format so as to besuited to the display scheme determined on the playback apparatus side,hence no consideration has been taken to make recording data versatile.

A 3-dimensional display involves various necessary information such asthe method of thinning of image, the number of viewpoints in a so-called“multi-view scheme” and the like other than the display scheme, theseinformation are not recorded as the recorded data when a single displayscheme is used. It is true that if only one identical display scheme isalways used, it is not necessary to record these information at all, butthe versatility of recording data is markedly reduced because of this.Just referring to the limited cases where data for the parallax barriersystem (or the lenticular system) is to be recorded, the left-eye imageand right-eye image may be recorded as separate sequences, the data maybe recorded as a mixed image in which the left-eye image and right-eyeimage are arranged horizontally half-and-half in one frame as shown inFIG. 18( c), or the data may be recorded as a composite image in whichthe left-eye image and right-eye image are interlaced on alternatevertical lines of one pixel as shown in FIG. 17( b). Naturally, data ofdifferent recording formats should be handled by different displayingprocesses, but since it is impossible to know the format of data fromthe recorded data, there is a problem in that it is impossible to knowhow the data should be processed for display when a third person getsthe data.

Further, in the prior art, no consideration has been taken for recordingimage data from different viewpoints independently from each other so asto facilitate readout and reproduction of a desired viewpoint imageonly.

In the prior art, no sufficient consideration has been given tointerchangeability with existing apparatus, either. Specifically, in asystem disclosed in Japanese Patent Application Laid-open 2001-337994,the display systems capable of interpreting additional information alonehave been handled, but the additional information is not useful fordisplay systems that cannot interpret it.

Moreover, when a 3-dimensional image based on the above prior art isenlarged or reduced, the amount of protrusion and the amount of depth ofthe 3-dimensional image change, hence there occurs a problem that adesired stereoscopic effect cannot be obtained.

Referring first to FIGS. 39 and 40, description hereinbelow will be madebriefly on the principle of a stereoscopic display for presenting astereoscopic view by displaying separate images for the left and righteyes. Both of these drawings are schematic top views showing cases wherea user having a binocular distance d is observing a stereoscopic display1.

Generally, suppose that d[m] represents the distance between eyes of auser, D[m] the distance from the user to stereoscopic display 1, W[m]the width of the display, P[dot] the resolution of the display and l(alphabetical letter l) [dot] the distance between left and rightcounterpart points of a 3-dimensional image.

Then, the amount of protrusion z[m] when a 3-dimensional image protrudesforward is given byz=(l×W/P)×D/(d+(l×W/P))  Eq. (1)

The amount of depth z[m] when a 3-dimensional image sets back is givenbyz=(l×W/P)×D/(d−(l×W/P))  Eq. (2)

The parallax θ is given byθ=tan⁻¹(1/2D)×2  Eq. (3)

With this stereoscopic display, when a 3-dimensional image is enlargedor reduced, the extent of disparity between the left and right imageschanges, hence the resultant image changes in stereoscopic effect. Thiswill be described referring to the 3-dimensional image beforeenlargement in FIG. 39( a) and the 3-dimensional image after enlargementin FIG. 39( b). When a 3-dimensional image having a protrusion forwardsfrom the stereoscopic display as shown in FIG. 39( a) is enlarged, theamount of protrusion becomes greater as shown in FIG. 39( b). Here, l′represents the left and right counterpart points after enlargement andz′ the amount of protrusion after enlargement.

On the other hand, when a 3-dimensional image having an setbackinterior-ward from the stereoscopic display as shown in FIG. 40( a) isenlarged and displayed, the amount of depth becomes greater, and withsome magnification ratio, it becomes impossible to present astereoscopic view because the views of the left and right eyes do notfuse. In contrast, when a 3-dimensional image is reduced in size, thedisparity between the left and right images becomes smaller, the amountof protrusion or the amount of depth becomes smaller, presenting a weakstereoscopic effect.

In this way, when a 3-dimensional image is enlarged or reduced, thestereoscopic effect varies as that protrusion becomes greater becausethe parallax becomes greater when the image is enlarged and converselydepth becomes smaller because the parallax becomes smaller when theimage is reduced. Therefore, if a 3-dimensional is enlarged or reducedin the same manner as a usual 2-dimensional image, there occurs aproblem that a desired stereoscopic view cannot be obtained causingconfusion or the uncomfortable stereoscopic view causes a strain on theeyes.

The present invention has been devised in order to solve the aboveproblems, it is therefore an object of the present invention to providea 3-dimensional image creating apparatus which can make the image datafor 3-dimensional display versatile and permits efficient selection ofan arbitrary viewpoint image as well as providing a 3-dimensional imagereproducing apparatus for reproducing the data.

It is another object of the present invention to provide a 3-dimensionalimage processing apparatus, a 3-dimensional image processing program anda recording medium recorded with the program, which can give warning tothe user and make correction so as to provide a comfortable stereoscopicview when the parallax quantity varies due to enlargement or reductionin size of a 3-dimensional image so that there is a fear that it becomesdifficult to obtain a stereoscopic view or the stereoscopic effect.

DISCLOSURE OF INVENTION

The present invention is a 3-dimensional image creating apparatuscomprising: a primary image creator for creating a primary image ofimage information for multiple viewpoints; a thumbnail image creator forcreating a thumbnail image; a 3-dimensional control information creatorfor creating 3-dimensional control information for implementing3-dimensional display of the primary image; and a multiplexer formultiplexing the primary image, the thumbnail image and the3-dimensional control information.

Here, the thumbnail image creator is characterized by creating thethumbnail image by directly reducing the primary image, creating thethumbnail image by extracting a section of one viewpoint image from theprimary image, embedding a symbol that indicates an inclusion of a3-dimensional image into the thumbnail image, or creating the thumbnailimage made up of a reduced image of the primary image and a reducedimage of one viewpoint image extracted from the primary image and fittedtherein in a picture-in-picture manner.

Further, the present invention is a 3-dimensional image reproducingapparatus, comprising: a demultiplexer for separating a primary imagedata, a thumbnail data and a 3-dimensional control information from aninput image data; and a thumbnail creator for outputting a thumbnailwith a symbol that indicates an inclusion of a 3-dimensional imageoverlaid on the thumbnail data when the primary image data represents a3-dimensional image.

The present invention is a 3-dimensional image processing apparatus,comprising: a parallax range acquisition means for acquiring a parallaxrange in which a stereoscopic view is permitted; a parallax quantityacquisition means for acquiring a parallax quantity of a 3-dimensionalimage; and a decision means for deciding whether the parallax quantityof the 3-dimensional image falls within the parallax range.

Further, a 3-dimensional image processing apparatus, includes: aparallax range acquisition means for acquiring a parallax range in whicha stereoscopic view is permitted; a parallax quantity acquisition meansfor acquiring a parallax quantity of a 3-dimensional image; a ratioacquisition means for acquiring a ratio for enlargement or reduction ofthe 3-dimensional image; and a decision means for deciding whether theparallax quantity of the 3-dimensional image that has been enlarged orreduced based on the ratio falls within the parallax range.

Here, the decision means is characterized by making a deciding processbased on a partial area of the 3-dimensional image.

The present invention is characterized by including a warning means forwarning a user or a parallax adjustment means for adjusting the parallaxquantity of the 3-dimensional image when the decision means determinesthat the parallax quantity falls out of the parallax area.

Here, the parallax quantity acquisition means is characterized by usageof a resolution and/or size of a stereoscopic display for displaying the3-dimensional image. Also, the parallax range acquisition means ischaracterized by usage of the capability of separating left and rightimages of a stereoscopic display for displaying the 3-dimensional image.Also, the parallax quantity acquisition means is characterized by usageof data previously tagged to the 3-dimensional image.

Further, the present invention is a 3-dimensional image processingprogram characterized by making the computer function as each of theaforementioned means.

Moreover, the present invention is a computer readable recording mediumhaving the above-described programs recorded therein.

According to the present invention, 3-dimensional control informationfor 3-dimensional display of the primary image is created, and theprimary image, its thumbnail image and the 3-dimensional image controlinformation are multiplexed, whereby it is possible to output thethumbnail image for efficient check of the image content when theprimary image is a 3-dimensional image.

According to the present invention, creating the thumbnail image bydirectly reducing the primary image makes it possible to provide displayof the thumbnail image in 3-dimension.

According to the present invention, creating the thumbnail image byextracting the section of one viewpoint image from the primary imagemakes it possible to display the thumbnail image without distortion.

According to the present invention, embedding the symbol that shows theinclusion of the 3-dimensional image into thumbnail images, enables evena conventional 3-dimensional image reproducing apparatus that cannotinterpret 3-dimensional control information, to make a selected filedistinctive as a 3D file from its thumbnail.

According to the present invention, creating the thumbnail image made upof a reduced image of the primary image and the reduced image of oneviewpoint image extracted from the primary image and fitted therein in apicture-in-picture manner, makes it possible to check both the imagecontent from the distortion-free image and the image configuration ofthe actual record of the primary image at the same time.

According to the present invention, when the data of the primary imagerepresents the 3-dimensional image, the image with the symbol thatindicates the inclusion of the 3-dimensional image overlaid on thethumbnail data is output as the thumbnail. It is thereby possible toshow whether the selected file is 2D or 3D from its thumbnail.

According to the present invention, since it is decided whether theparallax quantity of the 3-dimensional image falls within the parallaxrange for permitting the stereoscopic view, it is possible to takecountermeasures against the case when the stereoscopic view isimpossible. Examples of the countermeasures include warning to the userand the parallax quantity adjustment of the 3-dimensional image.Further, when the 3-dimensional image is enlarged or reduced, it isdecided whether the parallax quantity of the 3-dimensional image fallswithin the parallax range for permitting the stereoscopic view, and ifthe result shows difficulty in providing the stereoscopic view, it ispossible to take the measures.

Moreover, when the stereoscopic view is hard to achieve, it is possibleto adjust the parallax so as to provide as comfortable as possiblestereoscopic view by taking into account the parallax quantity (e.g.,the maximum amount of protrusion and maximum amount of depth) from thewhole or part of the 3-dimensional image and the parallax range thatpermits comfortable stereoscopic view on the stereoscopic display.

As described heretofore, according to the present invention, the user isallowed to properly check the content of 3-dimensional images and theuser is presented with the content of 3-dimensional images even if theyare enlarged or reduced, whereby it is possible to provide the3-dimensional image reproducing apparatus or image data processingapparatus which allows for comfortable observation of 3-dimensionalimages.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 is a diagram showing a 3-dimensional image creating apparatus inthe first embodiment.

FIG. 2 is a diagram showing a setup example of imaging devices formultiple viewpoints.

FIG. 3 is a diagram showing examples of how viewpoint numbers areallotted.

FIG. 4 is a diagram showing examples of joining for two viewpoints.

FIG. 5 is a diagram showing grid-like placements of multi-view images.

FIG. 6 is a diagram showing a format example of 3D information.

FIG. 7 is a diagram showing file formats of image data.

FIG. 8 is a diagram showing one format example when image data is storedin a file of an existing format.

FIG. 9 is a diagram showing one format example when image data is storedin a file of a new format.

FIG. 10 is a diagram showing a storage example of image data when imagedata at multiple viewpoints are recorded in separate files.

FIG. 11 is a diagram showing a 3D information format example.

FIG. 12 is a diagram showing one example of 3D information set values.

FIG. 13 is a diagram showing a management information format example.

FIG. 14 is a diagram showing one example when image data at multipleviewpoints are recorded in separate files.

FIG. 15 is a diagram showing a configuration of a 3-dimensional imagereproducing apparatus in the third embodiment.

FIG. 16 is a diagram showing an image display format in analternating-field system.

FIG. 17 is a diagram for illustrating the concept of a parallax barriersystem.

FIG. 18 is a diagram for illustrating the image display format in aparallax barrier system.

FIG. 19 is a diagram showing a configuration of a 3-dimensional imagecreating apparatus in the second embodiment.

FIG. 20 is a diagram showing a configuration of a 3-dimensional imagereproducing apparatus in the fourth embodiment.

FIG. 21 is a diagram showing image file formats for recording thumbnailimage data for 3-dimensional display.

FIG. 22 is a diagram showing combinations of a primary image and athumbnail image, reduced to 160 pixels×120 pixels.

FIG. 23 is a diagram showing thumbnail images in which a symbol thatindicates the inclusion of 3D image data is embedded.

FIG. 24 a diagram showing combinations of a primary image and athumbnail image in which a symbol that indicates the inclusion of 3Dimage data is embedded.

FIG. 25 is a diagram showing examples of thumbnails represented in apicture-in-picture manner.

FIG. 26 is a diagram showing a configuration of a 3-dimensional imagereproducing apparatus for reproducing files recorded with a thumbnail inthe sixth embodiment.

FIG. 27 is a diagram showing a configuration of a 3-dimensional imagereproducing apparatus which is switchable between 3D display and 2Ddisplay in the seventh embodiment.

FIG. 28 is a diagram showing a configuration of a 3-dimensional imagecreating apparatus which records a thumbnail image into a file in thefifth embodiment.

FIG. 29 is an illustrative diagram showing a GUI image frame of a3-dimensional image in the eighth embodiment.

FIG. 30 is a flowchart showing the process in the eighth embodiment.

FIG. 31 is an illustrative diagram showing a GUI image frame of a3-dimensional image in the ninth embodiment.

FIG. 32 is a flowchart showing the process in the ninth embodiment.

FIG. 33 is an illustrative diagram showing the parallax quantity of a3-dimensional image in a correction process.

FIG. 34 is a diagram for explaining correction of stereoscopic effect byvarying the amount of displacement between left and right images.

FIG. 35 is a flowchart showing the process in the tenth embodiment.

FIG. 36 is a flowchart showing the process at Step 17 in the tenthembodiment.

FIG. 37 is a diagram for explaining the method of changing the area foracquiring the parallax quantity.

FIG. 38 is a block diagram showing a 3-dimensional image processingapparatus in the eleventh embodiment.

FIG. 39 is a diagram for explaining protrusion of a 3-dimensional image.

FIG. 40 is a diagram for explaining depth of a 3-dimensional image.

BEST MODE FOR CARRYING OUT THE INVENTION

The embodiments of the present invention will hereinbelow be describedwith reference to the drawings.

The First Embodiment

FIG. 1 is a block diagram showing a configuration of a 3-dimensionalimage creating apparatus in accordance with the first embodiment. InFIG. 1, a 3-dimensional image creating apparatus 100 includes: an imagejoining unit 101 for determining the placement mode of images 1 to Kviewed from multiple viewpoints (K represents the number of viewpoints,here K is an integer equal to or greater than 2) and generating anjoined image by joining these images contiguous to each other; acontroller 102 for selecting whether images 1 to K are to be joined(presence/absence of joining), whether images 1 to K are to be reducedin size (presence/absence of reduction), one image to be used for2-dimensional display (2D select) and the number of viewpoints and theimage placement order; a 3D information creating unit 103 for creating3D information by formatting the presence/absence of reduction, thepresence/absence of joining, the image placement mode, the 2D select andthe information of the number of viewpoints; and a multiplexer 104having a means for having access to recording media and communicationslines and multiplexing image information and 3D information andoutputting the image data.

The operation of the thus configured 3-dimensional image creatingapparatus 100 will be described.

An image signal composed of continuous frames is input to the3-dimensional image creating apparatus, frame by frame. Here, imagingdevices for input of images to 3-dimensional image creating apparatus100 are arranged in a gridlike fashion within a plane, with M unitshorizontally and N units vertically, and each imaging device is allottedwith a number (viewpoint number) (here, M and N are integers equal to orgreater than 1).

FIG. 2 shows a setup example of 8 viewpoints (a view in which thearrayed imaging devices are viewed from the above rear). Here, theviewpoint numbers are assigned from left to right and top to bottom.Specifically, 1 is assigned to imaging device 301, 2 to imaging device302, 3 to imaging device 303 and 4 to imaging device 304. Similarly, 5to 8 are assigned to imaging devices 305 to 308, respectively. In allthe embodiments, the image taken by the imaging device designated by theviewpoint number k should be called image k (k is an integer equal to orgreater than 1).

Controller 102 designates the presence/absence of reduction, thepresence/absence of joining, 2D select, the number M of viewpoints inthe horizontal direction and the number N of viewpoints in the verticaldirection and the image placement order. Here, the presence/absence ofreduction takes a value for “reduced” or “no reduction”, and thepresence/absence of joining takes a value for “unjoined” or “joined”.The 2D select takes a value of the viewpoint number or a value for “nonedesignated”. As to the image placement order, the order of images isdesignated with the viewpoint numbers. The number of viewpoints shouldhave M=4 and N=2, in the example of FIG. 2.

It is assumed that when the presence/absence of joining, input fromcontroller 102 indicates “unjoined”, image joining unit 101 shouldoutput images 1 to K that have been input in parallel, sequentially inaccordance with the image placement order designated by controller 102.Alternatively, it is also possible to set up such that the output of theviewpoint images is always started from that of the viewpoint numberdesignated by the 2D select.

Image joining unit 101 selects placement mode of input images 1 to Kwhen the presence/absence of joining indicates “joined”. There can bethree kinds of placement modes: horizontal placement in which multi-viewimages are arranged horizontally; vertical placement in which multi-viewimages are arranged in the top and bottom direction; and gridlikeplacement in which multi-view images are arranged in both the horizontaland vertical directions.

Here, the placement mode of the images may or may not be in agreementwith the setup manner of the imaging devices. When the placement of theimages is in agreement with the setup manner of the imaging devices, M=1and N≧2 produces vertical placement, M≧2 and N=1 produces horizontalplacement, and other cases produce gridlike placement. For the caseswhere the placement is not in agreement, it is possible to adapt suchthat either the vertical placement or the horizontal placement can beselected when M=1 and N≧2 or M≧2 and N=1.

Once the placement mode is determined, the images are joined inaccordance with the image placement order input from controller 102.FIG. 3 shows examples of image placement orders when the images taken byimaging devices shown in FIG. 2 are arranged in a gridlike fashion. InFIG. 3, each cell represents one image and the numerals indicateviewpoint numbers. FIG. 3( a) shows a case where the image placementorder is designated to be 1, 2, 3, 4, 5, 6, 7 and 8, or the same orderas that of the viewpoint numbers assigned to the imaging devices. FIG.3( b) shows a case where the image placement order is designated to be2,3, 1, 4, 6, 7, 5 and 8.

If the presence/absence of reduction input from controller 102 shows“reduced”, the input image of each viewpoint is reduced. Upon reduction,the reduction ratio is not fixed but should be determined based on thenumber of viewpoints. Specifically, the image is reduced to 1/M in thehorizontal direction and 1/N in the vertical direction.

FIGS. 4 and 5 show examples of the results of joining by image joiningunit 101. FIG. 4 shows the cases of two viewpoints, designated by, forexample, the viewpoint Nos. 1 and 2, of the imaging devices shown inFIG. 2. FIG. 5 shows a case where all the imaging devices are used.

FIG. 4( a) shows a case with “no reduction” and “unjoined”; FIG. 4( b)shows a case with “no reduction” and “joined (horizontal placement)”;FIG. 4( c) shows a case with “no reduction” and “joined (verticalplacement)”; FIG. 4( d) shows a case with “reduced” and “unjoined”; FIG.4( e) shows a case with “reduced” and “joined (horizontal placement)”;and FIG. 4( f) shows a case with “reduced” and “joined (verticalplacement)”. Here, for the cases with “reduced”, the horizontal orvertical resolution is reduced to half by thinning the pixels or thelike.

FIG. 5 shows examples of gridlike placement of multi-view images, inparticular, a case where the presence/absence of reduction indicates “noreduction” is shown in FIG. 5( a). Here, H and V represent the number ofpixels in the horizontal direction, and the number of lines in thevertical direction, of each viewpoint image before reduction. A casewhere the presence/absence of reduction indicates “reduced” is shown inFIG. 5( b). The reduction ratio is 1/4 with respect to the horizontaldirection and 1/2 with respect to the vertical direction, the imageafter reduction is composed of H pixels in width and V lines in height,so that it is the same in size as each viewpoint image before reduction.As to the image arrangement, the images are arranged in the same manneras the imaging devices are set up, with four images in the horizontaldirection and two images in the vertical direction. Here, the numeralsattached to the images indicate the viewpoint numbers. The images arearranged in the ascending order of the viewpoint number: the upper leftis the image (viewpoint No. 1) taken by imaging unit 301 and the lowerright is the image (viewpoint No. 8) taken by imaging unit 308.

Description herein was made with the reduction ratios in the horizontaland vertical directions fixed, but these may be varied. When thereduction ratio is variable, it should be recorded in the 3Dinformation. When the presence/absence of joining indicates “unjoined”,the reduction ratio may be designated for every image at each viewpoint.

Three-dimensional information creating unit 103 creates 3D informationby formatting the presence/absence of reduction, the presence/absence ofjoining, 2D select, the number of viewpoints in the horizontal directionand the number of viewpoints in the vertical direction, the imageplacement order and the placement mode.

FIG. 6 shows one example of 3D information of this case. Here, the orderof images indicates that the images are arranged either in “viewpointnumber order” or “arbitrary order”. After this, a plurality of viewpointnumbers are recorded. These viewpoint numbers indicate the way theimages are arranged in the joined image when the presence/absence ofjoining indicates “joined”. In the example shown in FIG. 3( b), thefirst viewpoint number is 2 and followed by 3, 1, 4, 6, 7, 5 and 8. Whenthe presence/absence of joining shows “unjoined”, the order indicatesthe order in which the images of information are multiplexed. When theorder of images indicates “viewpoint number order”, these viewpointnumbers can be omitted. To create 3D information, the set values may beused as they are, or may be encoded by fixed length coding or byvariable length coding.

Multiplexer 104 converts image information, 3D information andmanagement information into data in a predetermined format and outputsit to the outside. When the images are not joined, the order of outputfrom image joining unit 101 is in agreement with the image placementorder designated by controller 102 as stated above. Therefore, the imageinformation is also multiplexed in the image placement order. Though notillustrated in FIG. 1, if voice sound and music are to be multiplexed,data for these is also multiplexed at multiplexer 104.

The Second Embodiment

Here, input to multiplexer 104 may be coded image information. FIG. 19shows a configuration of a 3-dimensional image creating apparatus 110dealing with this case. The 3-dimensional image creating apparatus 110differs from the 3-dimensional image creating apparatus 100 shown inFIG. 1, in that it includes an encoder 105.

The output from multiplexer 104 is connected to recording devices suchas IC memories, magneto-optical disks, magnetic tape, hard disks and thelike, and/or communications devices such as LAN, modems and others.Here, it is assumed that an IC memory is connected to multiplexer 104.Next, the recording format used in this case will be described.

Generally, when an IC memory is used as a recording medium, a filesystemsuch as FAT (File Allocation Table) etc., is constructed on the ICmemory and data is recorded as a file.

As the file format used herein an existing one or a newly defined uniqueformat may be used.

FIG. 7 is a diagram showing a file format for recording image data. InFIG. 7, it is assumed that data is recorded into a file, in the orderfrom the top to bottom in the figure. FIG. 7( a) shows a case where anexisting format is used; and FIG. 7( b) shows one example where a newformat is used.

When an existing format is used, 3D information is to be recorded aspart of the existing header portion, using a general function ofextending the header portion provided for the existing format. Here, theextended header is called an extension header. For example, a fileheader corresponds to an application data segment in JPEG, hence anewapplication data segment is defined to record 3D information. In MPEG-4,a file header corresponds to a Visual Object Sequence or/and VideoObject Layer, so that 3D information is recorded as user data intothese.

When an existing format is used, generally used extensions should beemployed directly. For example, an extension “.jpg” is used generally ina case of a JPEG file; an extension “.mpg” or “.mp4” is used generallyin a case of a MPEG file; and an extension “.wmv” is used generally in acase of WMV (Windows® Media Video). This makes it possible for even aconventional player having no 3D image display function to recognize thefile as a file of an existing format and display it as a 2-dimensionalimage.

On the other hand, when a new format is employed, 3D information may berecorded, for example, at the beginning of the file as shown in FIG. 7(b). Further, in order to have the file understood to be a new formatfile, a unique extension that makes it distinguishable from the files ofexisting formats should be added. The management information in FIGS. 7(a) and 7(b) should be used for recording some information such as dateof creation, creator and the like, which does not directly relate to the3-dimensional image.

To begin with, how multi-view images are stored when thepresence/absence of joining indicates “unjoined” will be described. Whena file having the existing format shown in FIG. 7( a) is used, aplurality of images viewed from multiple viewpoints are separatelyrecorded in the image information area in FIG. 7( a). When a motionpicture is recorded, a plurality of frames of data are recorded for eachviewpoint. FIG. 8 shows a stored example of this case. For a motionpicture, multiple frames of data are recorded for each viewpoint. Inthis case, each frame may be coded independently from one another as inMotion JPEG, or the differences may be coded by using inter-frameprediction as in MPEG-4.

A case where a file having a new format shown in FIG. 7( b) is used willbe described. In use of a new format, there are two cases, one whichemploys an existing format (JPEG, bitmap, etc.) for the file header andthe image information part in FIG. 7( b), and one which employs a quitenew unique format. Accordingly, in order to clarify the difference informat, classification information (called image type) should berecorded in the 3D information.

Referring to the way in which image data is recorded, a plurality ofimages data at multiple viewpoints are recorded in the image informationarea in FIG. 7( b). In a case of a motion picture, a plurality of framesof data are recorded for each viewpoint. FIG. 9 shows a storage exampleand 3D information example when the presence/absence of joiningindicates “unjoined”. K pieces of file headers and image information inFIG. 9 are formatted so that each part can be recognized as a file of anexisting format. When specifically describing it taking an example ofbitmap files, images 1 to K each are recorded as independent bitmapfiles, and the format is configured so that 3D information, managementinformation, the bitmap file of image 1 to the bitmap file of image Kare connected sequentially.

Here, when the presence/absence of joining indicates “unjoined”, imageinformation as to multiple viewpoints may be recorded as separate filesof image data. In this case, the image information for each viewpointcan be recorded in a format shown in FIG. 7( a) when using an existingformat and in a format shown in FIG. 7( b) when using a new format. Whenthe files are recorded, the existing format or new format alone may beused, or both the formats may be mixed and used. FIG. 10 shows anexample where an existing format is employed. Since there are Kviewpoints, K files are created.

In this case, 3D information creating unit 103 creates as much 3Dinformation as the number of viewpoints. Here, since one imageinformation is recorded in a single file, the order of images is omittedand the viewpoint number that shows the correspondence between the fileand the viewpoint number is to be recorded. FIG. 11 shows one example of3D information of this case.

FIG. 12 is a diagram showing one example of 3D information when thenumber of viewpoints is 2. Here, it is assumed that among the imagingdevices shown in FIG. 2, the imaging devices designated by viewpointNos. 1 and 2 are used. Since the number of viewpoints is 2, two 3Dinformation shown in FIGS. 12( a) and (b) are created. The left side of“=” shows the items of 3D information and the right side shows thecorresponding set values. Of all the items, the same values are recordedin both FIGS. 12( a) and 12(b), for the placement mode, the number ofviewpoints in the horizontal direction and the number of viewpoints inthe vertical direction and 2D select. Other items than these aredifferent: specifically, FIG. 12( a) shows that the image has aviewpoint number of 1 and is not reduced in size, and FIG. 12( b) showsthat the image has a viewpoint number of 2 and is reduced.

When image information at multiple viewpoints is recorded as separatefiles, among a number of recorded files, it is necessary to identify afile corresponding to a viewpoint from multi-view images taken by anidentical imaging apparatus. Here, it is possible to adapt multiplexer104 in FIG. 1, for example, to record the information for identifyingthe files of different viewpoint images taken by an identical sameimaging apparatus, into the aforementioned management information. FIG.13 is a diagram showing one example of management information of thiscase, wherein information that shows relationship between multi-viewimage files and the names recorded on the recording medium is recorded.In FIG. 13, the file configuration takes a value for “separated” or“integrated”: “separated” indicates that each of viewpoint images isrecorded as a separated file while “integrated” indicates that all theviewpoint images are recorded in one file. In order to acquire theknowledge of the filename of image data from the viewpoint number, eachviewpoint number is recorded in correspondence to the filename.

In addition, in order to show the multi-view images taken by the imagingdevices to belong to the same group, filenames can be assigned inaccordance with a predetermined naming rule. For example, in theaforementioned case having two viewpoints, a group consisting of image 1and image 2 may be assigned with filenames of “stereo1_1.jpg” and“stereo1_2.jpg”, and another group of images may be assigned withfilenames of “stereo2_1.jpg” and “stereo2_2.jpg”, so as to makedistinction from each other.

In addition, there is redundancy in the 3D information or the managementinformation recorded in the associated files. For instance, in theexample of 3D information as shown in FIG. 12, other than thepresence/absence of reduction and the viewpoint number, identicalinformation are presented. In the management information example sown inFIG. 13, the content of management information is common to all thefiles.

Accordingly, these common information (common information) may berecorded into a single file only and the other files may be adapted tobe recorded with the information (individual information) inherent toimage data of their own. FIG. 14 shows an example of this case. FIG. 14(a) shows a file having individual information and common informationrecorded therein. FIG. 14( b) shows a file having individual informationalone. In this example, the file of FIG. 14( a) is recorded in anexisting format and the file of FIG. 14( b) is recorded in a new format,so that the file extensions can be used to distinguish the file recordedwith common information from the other files recorded with individualinformation only. In addition, the filename of the file recorded withcommon information may be recorded as individual management informationas shown in FIG. 14( b). This structure facilitates to distinguish thefile recorded with common information from files recorded withindividual information alone.

Here, other than the above, distinction between these files can be alsoachieved by making the image file of each viewpoint distinguishable fromthe others using the aforementioned naming rule, and recording commoninformation to an image file having a particular viewpoint number, suchas that designated by the 2D select.

Alternatively, redundant information may be integrated to create amanagement file while each image file is adapted to be recorded withinherent information alone. The management file should have a uniqueextension different from that of the image files.

Additionally, a file system such as the aforementioned FAT usesdirectories so as to manage files en bloc. It is also possible to recorda group of created image files of different viewpoints (which mayinclude its management file if any) in the same directory.

When the presence/absence of joining indicates “joined”, the imageinformation of one integrated image is recorded in the image informationarea in FIG. 7( a) and FIG. 7( b). For a motion picture, a plurality ofimages, each made up of joined frames corresponding to multi-viewimages, are recorded.

Though the above embodiment is configured so that the placement order ofimages can be changed arbitrarily while the way of assignment of theviewpoint numbers is fixed, the way of assignment of the viewpointnumbers may be changed arbitrarily while the placement order of imagesis fixed. Further, the arrangement of imaging devices is not limited togridlike arrangements, but they can be arranged arbitrarily. In thiscase, a reference imaging device (whose viewpoint number is set at 1) isselected, and the position is expressed by a coordinate system havingthe position of the reference device as an origin. The positionalcoordinates of the imaging device at each viewpoint are recorded into 3Dinformation, in the order of viewpoint number.

In the above embodiment, the file of a new format is configured so that3D information and management information are recorded at the beginningof the file, however, files of a new format are not limited to this. Thestorage position of these may be located after the file header or afterimage information, or may be the same as the existing format shown inFIG. 7( a).

The Third Embodiment

Next, a reproducing apparatus for displaying the image data created by3-dimensional image creating apparatus 100 as 3-dimensional image willbe described.

FIG. 15 is a block diagram showing a configuration of a 3-dimensionalimage reproducing apparatus in accordance with the embodiment of thepresent invention. In FIG. 15, a 3-dimensional image reproducingapparatus 200 is comprised of a demultiplexer 201, a 3D informationanalyzer 202 and an image converter 203.

Demultiplexer 201 reads image data which has been multiplexed in thepredetermined format, from a recording device or a communications deviceand separates it into image information, 3D information and managementinformation. Though not illustrated in FIG. 15, when voice sound and/ormusic have been multiplexed, these data are also separated throughdemultiplexer 201.

Three-dimensional information analyzer 202 analyzes the 3D informationin the predetermined format and extracts the set value for each item.

Connected to image converter 203 may be various types of display deviceshaving different display formats, such as 2-dimensional display devicesusing ordinary CRTs, liquid crystal panels, stereoscopic display devicesusing lenticular technology, parallax barrier technology, time-divisionmethod, etc.

Concerning the thus configured 3-dimensional image reproducing apparatus200, its operation will be explained. Here, it is assumed that an ICmemory is connected to demultiplexer 201. As already stated, image filesin an existing format and in a new format as well as management filesare recorded in the IC memory. Distinction between image files andmanagement files can be made from their file extensions. Here, it isassumed that the user selects one image file or management file via anunillustrated selecting means.

To begin with, a case where the selected file is an image file will bedescribed. Since, in this case, distinction between the existing formatand the new format can be made from their file extensions, demultiplexer201 reads out 3D information from the extension area of the file headerwhen the file to be reproduced is an existing format file shown in FIG.7( a). When the file is that of a new format shown in FIG. 7( b), 3Dinformation is read out from the beginning of the file.

Three-dimensional information analyzer 202 analyzes the 3D informationand extracts the set values for the presence/absence of joining,presence/absence of reduction, the number of viewpoints, placement mode,2D select and others. It also determines the viewpoint numbers of theimages for displaying a 3-dimensional image. To display a 3-dimensionalimage from multi-view image data, images at two viewpoints presentingparallax should be selected from the multi-view images and may be put touse for the left-eye and right-eye images. For example, in case of theimages of data recorded by the imaging devices shown in FIG. 2, ahorizontally arranged pair of devices, such as 1 and 2, 1 and 3, 1 and4, 2 and 3, 2 and 4, and others, can be selected. If the images arerotated 90 degrees upon display, pairs of devices 1 and 5, 2 and 6, 3and 7 and 4 and 8, may be selected to achieve display of a 3-dimensionalimage.

When the presence/absence of joining indicates “unjoined”, the selectedfile is either recorded with image information at one viewpoint only asshown in FIG. 10 or with unjoined separate image information for all theviewpoints as shown in FIG. 8.

Distinction between these can be done by checking whether the 3Dinformation contains the information as to the order of images, based onthe analyzed result of the 3D information. If the 3D informationcontains the order of images, the file is the latter one, and from thefirst to K-th viewpoint numbers recorded in the 3D information, anarbitrary combination of viewpoints allowing for a stereoscopic viewshould be selected. Otherwise, the file is the former one, and anotherviewpoint number which is able to produce a stereoscopic view incombination with the viewpoint number i (here i is an integer equal toor greater than 1) recorded in the 3D information is selected. Theselected viewpoint numbers are output to demultiplexer 201.

Demultiplexer 201 reads out the image information designated by theinput viewpoint numbers from the file, and outputs it to image converter203. If the image information at the input viewpoint numbers is notfound in the file, a file that has the image information recordedtherein should be located with the aid of the management information andnaming rule as stated above and read out.

It is also possible to refer the file configuration in the managementinformation in order to check whether the file has a record of only oneviewpoint image or a record of images of all viewpoints. If the fileconfiguration shows “separated”, image information of one viewpointimage is recorded and if it shows “integrated”, image information of allviewpoints is recorded.

On the other hand, when the presence/absence of joining indicates“joined”, the file has a record of only a single joined image, it istherefore possible to select arbitrary viewpoint numbers. In this case,the viewpoint numbers are output to image converter 203.

The Fourth Embodiment

At this point, when image information has been coded, the data should bedecoded after demultiplexing. FIG. 20 shows a configuration of a3-dimensional image reproducing apparatus 210 for this case.Three-dimensional image reproducing apparatus 210 differs from the3-dimensional image reproducing apparatus 200 in FIG. 15 by theinclusion of a decoder 204.

Image converter 203 converts the image information separated bydemultiplexer 201 into a display format, in accordance with thepresence/absence of joining, presence/absence of reduction, the numberof viewpoints, placement mode, 2D select and the viewpoint numbers,input from 3D information analyzer 202. In this process, if thepresence/absence of joining indicates “unjoined”, conversion will bestarted when all image information designated by the viewpoint numbershave been extracted. If any of the image information cannot be obtaineddue to file deletion or other reasons, the image designated at aviewpoint number closest to the original number may be used in place or,2-dimensional display may be implemented. When the presence/absence ofjoining indicates “joined”, the viewpoint images designated by theviewpoint numbers are cut out from the joined image and converted.

Referring to conversion into a display format, for example, if aparallax barrier display device is connected to 3-dimensional imagereproducing apparatus 200, the form shown in FIG. 4( e) is easiest tohandle. In this case, vertical lines of one pixel from the left andright images are rearranged alternately in the horizontal direction. Inthe case of FIG. 4( b), the left and right images are each thinned tohalf with respect to the horizontal before rearranging vertical lines ofone pixel alternately in the horizontal direction. In either case, theimage data is attached with 3D information, so that it is possible toconvert the data into a display format that is suitable for the displaydevice.

Here, taking an example of a parallax barrier system to refer the casewhere image information is encoded, the coding efficiency can bemarkedly improved when encoding is carried out for the image having anarrangement shown in FIG. 4( e), even if the actual image to bedisplayed is that shown in FIG. 17( b). This is because the state shownin FIG. 4( e) has a higher correlation between adjacent pixels than thatof the state shown in FIG. 17( b). To apply the encoded data of theimage of FIG. 4( e) to a parallax barrier system, in 3-dimensional imagereproducing apparatus 210 of FIG. 20 the data should be decoded into itsimage form by decoder 204 first, then the image should be rearranged byimage converter 203 into the form shown in FIG. 17( b).

If the operation mode is switched to the 2-dimensional display modewhile a 3-dimensional image is being displayed, the image of theviewpoint number designated by 2D select is displayed. At this point, ifthe presence/absence of reduction shows “no reduction”, the image isdirectly displayed. In the case of “reduced”, the image is enlargedtwofold and displayed. When, however, the image is not being displayed,one of the images being displayed may be selected and displayed. Thisselection can be done by, for example, selecting one with the least orgreatest viewpoint number, one that is closest in the distance, withrespect to the arrangement of the imaging devices or in the joinedimage, to the image of the viewpoint number designated by 2D select, onebeing used for the left-eye image, one being used for the right-eyeimage, or any other method. Here, the selection is not particularlylimited. If no image for 2-dimensional display is selected by 2D select,the image to be displayed is selected by a predetermined method.

When a 2-dimensional display apparatus is connected, the imagedesignated by 2D select is displayed. The display is done in the samemanner as 2-dimensional display in the 3-dimensional display device.

When the file selected by the user is a management file, thepresence/absence of joining recorded in the 3D information must show“unjoined”; an arbitrary combination of viewpoint numbers that allow fora stereoscopic view may and should be selected. The operation from thereadout of image data from the image files to the conversion into thedisplay format is the same as that described above, so that descriptionis omitted herein.

Now that the above embodiment was explained referring to a case wherethe viewpoint numbers and filenames are recorded in the managementinformation, it is also possible to provide a following configuration.That is, 3-dimensional image creating apparatus 100 is configured torecord an identification number that indicates that image informationbelongs to a group of multi-view images taken by the same imagingapparatus, into both the common information and the individualinformation, and 3-dimensional image reproducing apparatus 200 isconfigured to read out image information only when both the filename andidentification number are consistent. The same identification number maybe assigned to a group of multi-view images or a different number may beassigned to each viewpoint. This configuration can provide preventionagainst malfunction due to fabrication of a filename.

As stated heretofore, since various kinds of data created by different3-dimensional imaging schemes can be handled in a unified manner, andsince 2-dimensional images can be displayed correctly on conventionalplayers having no 3-dimensional image display function, it is possibleto provide versatility.

There are some cases where a thumbnail image is recorded in the imagefile as shown in FIG. 21( a). According to the DCF (Design rule forCamera File system) standard, established to ensure theinterchangeability of image files, in order to secure the leastinterchangeability for reproduction in case of failing to reproduce theprimary image (image information), storage of a thumbnail image isstipulated. In the DCF standard, no limitation is imposed on the pixelsize of the primary image, but the pixel size of the thumbnail image islimited to one kind, 160 pixels×120 pixels.

To record a file of information for 3-dimensional display along thisidea, the format shown in FIG. 21( b) can be considered. That is, thefile has a form in which 3D information is added to an image file havinga format of FIG. 21( a), and the primary image in FIG. 21( b) issupposed to be a 3D image.

Here, it is assumed that an image, as shown in FIG. 4( e), made up oftwo viewpoint images, each reduced to half in the horizontal directionand joined to each other, is stored as the primary image in FIG. 21( b),and that the pixel size is 640 pixels×480 pixels. It is also assumedthat an image directly reduced from the primary image, down to 160pixels×120 pixels, is used as the thumbnail image. Under the aboveassumption, the resultant file contains images as shown in FIG. 22( a).In this case, the primary image and the thumbnail image can be said tobe given in 3D image forms.

Accordingly, when a file with images as shown in FIG. 22( a) is given, a3-dimensional image reproducing apparatus capable of interpreting 3Dinformation and achieving 3D display, is able to provide a thumbnaildisplay as shown in FIG. 23( c), by overwriting the information thatindicates that the file holds 3D data, on the thumbnail image. It isalso possible to display the thumbnail in 3D by converting the thumbnailimage in the same manner as the primary image is converted when it isdisplayed in 3D. Moreover, when the symbol (the characters “3D” in FIG.23( c), for example) that indicates the inclusion of a 3D image may begiven with a parallax to achieve 3D display, it is possible to enhancethe visibility.

When a thumbnail is given in 3D display, use of a thumbnail image shownin FIG. 22( a) enables high-speed rendering compared to the case when adecoded primary image is reduced and used for 3D display. This isbecause a thumbnail image is small, hence can be decoded quickly, unlikethe case where a large image, i.e., the primary image, initially needsto be decoded.

Here, since it is not necessary that the thumbnail image is one that isdirectly reduced from the primary image, a combination of a primaryimage with a thumbnail image as shown in FIG. 22( b) is also possible.The primary image in FIG. 22( b) is the same as in FIG. 22( a) while thethumbnail image is an image, obtained by extracting the section of oneviewpoint image from the primary image and reduced down to 160pixels×120 pixels. In FIG. 22( b), since the primary image is contractedto half in the horizontal direction, or in other words, the ratiobetween the horizontal scale and vertical scale is 1:2, the thumbnailimage should be created by extracting the section of one viewpoint imageand enlarging it twice in the horizontal direction.

When a file with images as shown in FIG. 22( b) is given, a3-dimensional image reproducing apparatus capable of interpreting 3Dinformation and achieving 3D display, is able to provide a thumbnaildisplay as shown in FIG. 23( b), by overwriting the information thatindicates that the file holds 3D data, on the thumbnail image.Preparation of a thumbnail image by extracting the section of oneviewpoint image from the primary image as shown in FIG. 22( b) enablesquick thumbnail display free from distortion.

Even when a thumbnail image as shown in FIG. 22( a) is given, it ispossible to display a thumbnail as shown in FIG. 23(b) by extracting thesection of one viewpoint upon thumbnail display, magnifying it twice inthe horizontal direction, then overwriting the information for 3Dstorage indication, on the thumbnail image.

Though FIG. 22 shows a case where the presence/absence of joining in theprimary image indicates “joined”, the primary image may be composed of“unjoined” images as shown in FIG. 4( a). When the presence/absence ofjoining of the primary image shows “unjoined”, one of a plurality ofimages recorded for the primary image can be reduced and recorded as athumbnail image. Alternatively, whether the presence/absence of joiningof the primary image shows “joined” or “unjoined”, one viewpoint imagedesignated by the “2D select”, which was explained already, may beextracted to create its thumbnail image or provide thumbnail display.

As long as a 3-dimensional image reproducing apparatus is one that caninterpret 3D information as described heretofore, when receiving a fileas shown in FIG. 21( b) it can handle the file properly. Now, accountshould be taken of an old type 3-dimensional image reproducing apparatusthat cannot interpret 3D information. This 3-dimensional imagereproducing apparatus is one that has no 3D information analyzer 202 orno image converter 203 in FIG. 20 and is not connectable to a3-dimensional display. This 3-dimensional image reproducing apparatuscannot interpret 3D information, hence cannot tell whether the imageinformation stored in the file is for a 2D image or 3D image. Eventhough the file has been found to hold a 3D image by some unspecifiedmeans, it is impossible for the apparatus to reproduce the imageinformation correctly because it has no image converter.

Even in such a case, if thumbnail images have been recorded inaccordance with the aforementioned DCF standard or a similar idea, atleast display of the thumbnail images can be expected.

Thereupon, in the present invention, a thumbnail image embedded with asymbol indicating that the file contains a 3D image is recorded. Forexample as shown in FIG. 23( a), an image with a pictorial symbol “3D”overwritten at the lower right thereof may be used as a thumbnail image,or as shown in FIG. 23( b), an image embedded with transparent text “3D”in the center thereof may be used as a thumbnail image. Alternatively,if an image as shown in FIG. 23( c) is recorded as a thumbnail image, itis at least possible to recognize that the file is one that contains a3D image.

When a symbol that represents 3D content is embedded in the thumbnailimage, it is possible to permit the user to select the position and sizeof the symbol to be embedded at the time of recording. Further, it isalso possible to allow for selection of a desired symbol from aplurality of symbols prepared beforehand. Moreover, a background area isautomatically located upon creation of a thumbnail image, so that asymbol may be written into the background area. In addition, when thesymbol for representing 3D content is automatically embedded in thethumbnail image as stated above, it is possible to permit the user tocheck the once recorded thumbnail image and recreate the thumbnail imageif the position and/or size of the symbol is unpreferable.

FIG. 24 shows combination examples of the primary image and itsthumbnail image of an image file thus recorded. Here, similarly to thecase of FIG. 22, it is assumed that a joined image composed of twoviewpoint images, reduced to half in the horizontal direction as shownin FIG. 4( e) is stored as the primary image and that its pixel size is640 pixels×480 pixels. Also, it is also assumed that an image, reducedfrom the primary image, down to 160 pixels×120 pixels and embedded witha symbol indicating the inclusion of a 3D image, is used as a thumbnailimage. With this assumption, storage of a thumbnail image as shown inFIG. 23( c) produces a combination as shown in FIG. 24( a) and storageof a thumbnail image shown in FIG. 23( b) produces a combination shownin FIG. 24( b).

The symbol in itself may be text, a mark or particular image as long asit can indicate the existence of 3D content. Also, there is nolimitation on the position of the symbol in the thumbnail image.However, in either case, the symbol has to be recorded as part of thethumbnail image. Conversely, the symbol is not one that is recordedseparately from the thumbnail image.

This is the key. In order to provide effective thumbnail display in a3-dimensional image reproducing apparatus that cannot interpret 3Dinformation, it is necessary to record an image shown in FIG. 23, inadvance, in the file, as a thumbnail image. This arrangement makes itpossible for the 3-dimensional image reproducing apparatus to allow forcheck of image content of 3D image files in the same manner as 2D imagefiles by reproducing their thumbnail images, hence discriminate between2D image files and 3D image files.

Incidentally, even a 3-dimensional image reproducing apparatus capableof interpreting 3D information does not always support all kinds of dataformats. For example, a 3-dimensional image reproducing apparatus thatsupports only 3D images of a four-lens system cannot provide correctdisplay if it receives 3D image data of a twin lens system. In such acase, the image content can be checked by display of thumbnail images.Since a 3-dimensional image reproducing apparatus capable ofinterpreting 3D information is able to judge whether the primary imagecontained in a file is correctly reproducible, by analyzing the 3Dinformation, the apparatus can be configured to display a thumbnailimage if a file has an invalid data format which cannot be reproducedcorrectly and also display a message or the like, informing that “thefile has an unsupported 3D data format”.

The Fifth Embodiment

Now, FIG. 28 shows the example of a 3-dimensional image creatingapparatus for recording a thumbnail image in a file.

A 3-dimensional image creating apparatus 120 in FIG. 28 has a thumbnailimage creator 106 and creates a file by multiplexing thumbnail datatogether with coded data (or uncompressed image information) and 3Dinformation in a multiplexer 104. The output of the file can bedelivered to recording devices such as IC memories, magneto-opticaldisks, magnetic tape, hard disks and the like, and/or communicationsdevices such as LAN, modems and others. Though the size of a thumbnailimage in the DCF standard is defined as 160 pixels×120 pixels, generalthumbnail images are not limited to this size.

The Sixth Embodiment

Next, FIG. 26 shows the example of a 3-dimensional image reproducingapparatus for implementing thumbnail display as shown in FIG. 23 fromthe thumbnail images as shown in FIG. 22. In FIG. 26, the componentshaving the same functions as those in the 3-dimensional imagereproducing apparatus 210 in FIG. 20 are allotted with the samereference numerals and the description is omitted. In a 3-dimensionalimage reproducing apparatus 220 as shown in FIG. 26, thumbnail data isseparated by a demultiplexer 221. If the data to be reproducedrepresents a 3D image, a thumbnail creator 225, in accordance with theinstruction from a 3D information analyzer 202, implements thumbnaildisplay of a thumbnail image, decoded from the thumbnail data (nodecoding is needed if data is uncompressed) by superimposing a symbolindicating the fact of a 3D image. If the thumbnail is displayed in 3D,thumbnail creator 225 implements the same process on the thumbnail imageas the way in which image converter 203 makes conversion of the primaryimage.

The Seventh Embodiment

FIG. 27 shows the example of a 3-dimensional image reproducing apparatuswhich can output a 3D display image and a 2D display image in aswitchable manner. In FIG. 27, the components having the same functionsas those in the 3-dimensional image reproducing apparatus 220 in FIG. 26are allotted with the same reference numerals and the description isomitted. In a 3-dimensional image reproducing apparatus 230 shown inFIG. 27, when encoded image data of the primary image and thumbnailimage as shown in FIG. 24 is input, the image information decoded bydecoder 204 is sent to image converter 203 as well as to thumbnailcreator 225. A controller 226 gives instructions of whether a 3D displayimage or 2D display image is to be output, to image converter 203 andthumbnail creator 225. When the thumbnail is to be displayed in 2D,thumbnail creator 225 may directly output the thumbnail image as shownin FIG. 24 (in this case, the symbol representing 3D is also displayed)or may output a thumbnail image without any symbol indicating the factof 3D by reducing the input primary image and implementing a similarprocess to that of image converter 203. If the thumbnail is displayed in3D, the input primary image may and should be reduced and then subjectedto the same process as in image converter 203.

In the description heretofore, a 3D file is configured so that itsthumbnail image is displayed with a superimposed symbol that indicatesthe fact of 3D. However, it is possible to display the thumbnail imageas it is for a 3D file while display of the thumbnail image of a 2D filemay be performed by superimposing a symbol that indicates 2D.

Further, not only display of a symbol that indicates the 3D over thethumbnail image of a 3D file, but also display of the data contained inthe 3D information, such as the number of viewpoints, viewpoint numbersand others, may be superimposed on the thumbnail image. Moreover,display of the data contained in 3D information and the symbolindicating the fact of 3D may be superimposed on the thumbnail image butmay be given at a predetermined position near the thumbnail image.

FIG. 25 shows other examples of thumbnail images stored in the file. Thethumbnail image is composed of a first image made up of two or moreimages from different viewpoints and a second image that is created fromthe first image by cutting out one viewpoint image and the two are thenfitted in a picture-in-picture manner. In FIG. 25( a), the first imageis to be a main-image and the second image is to be a sub-image. Asshown in FIG. 25( b), the main-image and sub-image can be interchanged.FIG. 25( c) and FIG. 25( d) show the respective images of FIG. 25( a)and FIG. 25( b), each embedded with a symbol that indicates the fact of3D. As to FIGS. 25( c) and (d), the symbol indicating the fact of 3D maybe output and superimposed upon display of thumbnails, instead of itsbeing recorded in the thumbnail image. Use of thumbnail display as shownin FIG. 25 allows for both checking of the image content with adistortion-free image and confirmation of the actual image form recordedas a primary image.

The embodiment of the present invention will be described hereinbelowwith reference to the drawings.

The Eighth Embodiment

In the eighth embodiment of the present invention, using GUI applicationsoftware for 3-dimensional image display, a personal computer (to beabbreviated as PC, hereinbelow) implements a stereoscopic displayprocess to implement stereoscopic display on a stereoscopic display.

Specifically, the CPU on the PC performs processing of a motion pictureand/or a still image, in accordance with the stereo display applicationsoftware recorded on a recording medium such as a CD-ROM, hard disk andthe like, to implement stereoscopic display on the stereoscopic display.Further, as the user gives instructions for stereo processing through amouse or a keyboard, the CPU executes the process based on theinstructions.

FIG. 29 is a diagram for explaining a display image on a stereoscopicdisplay of the eighth embodiment, in which a management display image 2is displayed on display 1 by the 3-dimensional image displayapplication. Management display image 2 of the 3-dimensional imagedisplay application is composed of a 3-dimensional image display area 3,a magnification ratio adjustment bar 4 and a warning display area 5.

As understood from FIGS. 39 and 40 used for description of the priorart, when a 3-dimensional image is enlarged and displayed, both theamount of protrusion projected forward from the stereoscopic display andthe amount of depth depressed interior-ward with respect to thestereoscopic display are magnified, and when they become greater thantheir corresponding certain thresholds, it is impossible to produce astereoscopic view. At this occasion, if the maximum amounts ofprotrusion and depth of the 3-dimensional image are known in advance, itis possible to determine to what extent the amounts of protrusion anddepth of the 3-dimensional image change depending on the enlargementratio, from Eqs. (1) and (2). In the present embodiment, if, forexample, protrusion of the 3-dimensional image becomes stronger andhence a long watching-hours could cause a burden on the user, a warningis given to the user.

Referring next to a flowchart shown in FIG. 30, the processing flow ofthe eighth embodiment will be described.

At Step S1, display information is acquired. The display informationmentioned here includes width W[m] of the display, resolution P[dot] ofthe display and distance D[m] between the user and the display. Thedistance between the user and the display may be obtained exactly usinga position sensor or the like. However, it is more convenient to use avalue from a database or the like, in which general distances to theuser related to the sizes and types of the display, have beenaccumulated beforehand, as an example the distance to the user maybe asmuch as 1 meter if the display size is 15 inches.

As to the stereoscopic displays for producing stereoscopic effect byprojecting different images on the left and right eyes, there occurs nomixture between the left and right images when absolutely independentdisplay devices are used for the left and right eyes as in the case ofHMD. However, in stereoscopic displays based on the parallax barriersystem and lenticular system, a slight component of the image for theuser's right eye may mix with the image to be viewed by the left eye.This is called crosstalk. In general, the less crosstalk, the moreexcellent the stereoscopic display is. As crosstalk becomes greater, therange for permitting the comfortable stereoscopic view becomes narrower.For this reason, information as to crosstalk is included into thedisplay information, and when crosstalk is strong as will be describedhereinbelow, the range of parallax may be made small so as to providecomfortable stereovision on the stereoscopic display.

For this purpose, a parallax range for permitting the stereoscopic viewis acquired at Step S2.

The CPU of the PC calculates the parallax quantity θ in the range forpermitting the comfortable stereoscopic view based on the disparitybetween the left and right images, using Eqs. (1), (2) and (3), so as todetermine the amount of protrusion th_f [dot] and the amount of depthth_b [dot] falling in the range for permitting the comfortablestereoscopic view. Here, th_f and th_b are represented by the distances(parallax) between left and right counterpart points on the stereoscopicdisplay screen. As apparent from FIGS. 39 and 49, in an image thatprotrudes forward from the screen, the left-eye image is located to theright with respect to the right-eye image whereas the left-eye image islocated to the left with respect to the right-eye image, in an imagethat sets back from the screen. Hence, in the present invention, theextent of disparity of the right-eye image from the left-eye image as areference is determined so that a positive value of parallax willrepresent a protrusion (the left-eye image located on the right) and anegative value of parallax will represent a depth (the left-eye imagelocated on the left).

The range in which the value of parallax falls from th_f to th_b, can besaid to be the maximum parallax range of stereoscopic display forpermitting the comfortable stereoscopic view, and th_f and th_brepresent the thresholds of the maximum parallax range.

The parallax quantity θ for permitting the comfortable stereoscopic viewhas been empirically determined by various studies and is known that ithas correlation with the size of the stereoscopic display and crosstalk.For example, when it is assumed that the range of θ that permits thecomfortable stereoscopic view in a 15-inch stereoscopic display is 35minutes for both protrusion and depth and the distance between user'seyes is 60 mm, the thresholds that specify the maximum parallax rangefor stereoscopic display are about th_f=25 and th_b=−25 [dot].Accordingly, the parallax range that permits the stereoscopic view is−25≦θ≦25. Naturally, these values change depending on the information asto the display acquired at S1. In addition, since these values aredetermined at best empirically, instead of using Eqs. (1), (2) and (3),it is naturally considered that CPU may use the experimentally obtainedmeasurement of display performance of stereoscopic displays, which hasbeen stored beforehand in a database or the like.

At Step S3, the maximum amount of protrusion f [dot] and maximum amountof depth b [dot] of a 3-dimensional image to be displayed, in one word,the maximum parallax quantity of a 3-dimensional image is acquired fromthe tag information of the 3-dimensional image. Here, tag informationrefers to additional information, separately attached to a 3-dimensionalimage, such as conditions of shooting when the image was taken. In thepresent embodiment, it is assumed that the information on the maximumamount of protrusion and maximum amount of depth of a 3-dimensionalimage is given in advance as tag information. However, there is a methodof automatically determining the information on f and b by stereomatching, meaning that the method of acquisition is not limited to useof tag information. Also in this case, f and b are represented as theparallax quantities on the stereoscopic display screen. These valuesdenote the range of parallax (parallax range) of a 3-dimensional image.

At Step S4, the user inputs a ratio E [%] of enlargement or reduction ofthe 3-dimensional image by means of magnification ratio adjustment bar4. Input of the magnification ratio is not limited to use of themagnification ratio adjustment bar, but it can be modified by pressingparticular board keys, or by use of a scroll bar with a mouse. Inaddition, the present invention is supposed to not only deal with anenlargement process but also a reduction process, and the ratio forenlargement or reduction is supposed to be obtained at Step S5.

At Step S5, whether the parallax quantity of a 3-dimensional image fallswithin the parallax range that permits the stereoscopic view is decided.The maximum amount of protrusion f and maximum amount of depth b, bothdetermined at Step S3, are multiplied by the magnification ratio E [%]obtained at Step S4, so as to determine the maximum amount of protrusionf′=Ef [dot] of the enlarged 3-dimensional image and the maximum amountof depth b′=Eb [dot] of the enlarged 3-dimensional image. The resultantvalues are compared to th_f and th_b determined at Step S3 to decidewhether these values fall within the parallax range for permitting thestereoscopic view, at Step S5. If the parallax quantity of the3-dimensional image falls within the above parallax range, the processis ended. If the parallax quantity of the 3-dimensional image falls outof the parallax range, the operation goes to Step S6.

At Step S6, the CPU of the PC determines a message, such as “too strongprotrusion”, “too strong depth”, “too weak protrusion”, “too weak depth”or the like, based on what degree the parallax quantity of the3-dimensional image is off the parallax range for permitting thestereoscopic view, and displays it on warning display area 5. Here,warning to the user may be given by a voice message, by changing thecolor of 3-dimensional image display area 3 or by any other method, andshould be limited to display of the message on warning display area 5.

Though, in the present embodiment, decision on the amount of protrusionof a 3-dimensional image is made based on the parallax quantity [dot] orthe disparity between the counterpart points of the left and rightimages, obviously, the amount z [m] of the apparent protrusion from thedisplay as shown in FIGS. 39 and 40 may be used for this decision.

The Ninth Embodiment

The ninth embodiment of the present invention will be described.

A management display image 6 of stereoscopic display 1 in the ninthembodiment is composed of a 3-dimensional image display area 3, amagnification ratio adjustment bar 4 and a warning display area 5, asshown in FIG. 31.

The operational flow of the ninth embodiment will be described referringto the flowchart of FIG. 32. Step S1 to Step S5 are identical with theeighth embodiment.

At Step S16, correction to the amount of protrusion (parallax quantityadjustment) is made as shown in FIG. 33. FIG. 33( a) shows the parallaxrange of a 3-dimensional image before enlargement, (b) the parallaxrange of a 3-dimensional image after enlargement and (c) the parallaxrange of a 3-dimensional image after a correction process. In FIG. 33,the lateral axis represents the size of the disparity between stereocounterpart points on the left and right images, and hatching representsthe range of parallax in which the comfortable stereoscopic view isobtained. Concerning the symbols, f and b represent the maximum amountsof protrusion and depth of a 3-dimensional image before enlargement, andf′ and b′ represent the maximum amounts of protrusion and depth of theimage after enlargement, and th_f and th_b represent the maximum amountsof protrusion and depth, between which the comfortable stereoscopic viewcan be obtained on the stereoscopic display. Even if the maximum amountof protrusion f′ of the image after enlargement is greater than th_f, iff′−b′ is smaller than th_f_th_b, the right-eye image as a whole may beshifted by f−th_f′ as shown in FIG. 34 so as to set back the apparentposition of the 3-dimensional image as a whole with respect to thestereoscopic display, to thereby correct stereoscopic effect. In FIG.34, part 6 enclosed by the broken line represents the original image andpart 7 enclosed by the solid line represents the image after shifting.However, if f′−b′ is greater than th_f−th_b, it is no longer possible tocorrect the protrusion by simply shifting the right-eye image as awhole.

Similarly, in the case of handling depth, when the maximum amount ofdepth b′ after enlargement of the image is smaller than th_b and f′−b′is smaller than th_f−th_b, shifting the right-eye image as a whole byb′−th_b to display the entire 3-dimensional image being projected fromthe stereoscopic display, makes it possible to provide a comfortablestereoscopic view. It should be noted that when the amount of shiftingis positive, the right-eye image as a whole is shifted to the right, andwhen the amount of shifting is negative, the right-eye image as a wholeis shifted to the left.

In the present embodiment, though the amount of protrusion of the whole3-dimensional image is corrected by shifting the right-eye image, theway of shifting is not limited to that of the right-eye image but theleft-eye image may be shifted with the right-eye image fixed or both theimages are shifted at the same time.

Further, when f′−b1 is greater than th_f−th_b, display of “TOO STRONG TOCORRECT” in the warning area or display of one of the left and rightimages in the 3-dimensional image display area makes it possible to showthe user an uncomfortable 3-dimensional image, though provision ofwarning display area 5 or any other warning is not compulsory.

The embodiment is not limited to the correction of stereoscopic effectupon enlargement, it also possible to enhance the stereoscopic effect ofa 3-dimensional image by making the image protrude or set back as awhole when the 3-dimensional image lacks stereoscopic effect as a resultof size reduction. For example, when the magnification ratio is smallerthan 1 at Step S4, the 3-dimensional image after reduction is shifted byf−f′ at Step S16 so as to make the image project forwards with respectto the display and equalize the position of the maximum protrusion afterreduction with that before reduction, whereby it is possible to make the3-dimensional image after reduction give a greater feeling of protrusionor protrude greatly forward from the screen. Conversely, it is alsopossible to equalize the amount of depth so that the maximum position ofprotrusion does not remain fixed and the maximum position of depth isfixed.

The Tenth Embodiment

The tenth embodiment of the present invention will be described.

The tenth embodiment of the present invention is an improvement of theninth embodiment, in which the protrusion correction process (parallaxquantity adjustment) at Step S16 is improved.

This embodiment enables display for easy observation of a stereoscopicview by adjusting the stereoscopic effect giving priority to the centralpart of the 3-dimensional image even when a comfortable stereoscopicview cannot be obtained by the total shift of a 3-dimensional image.This embodiment takes advantage of the characteristic of the human'ssense of vision, or the fact that the human's vision is clearer aroundthe center than the periphery of the viewfield.

The tenth embodiment is the same as the ninth embodiment from Steps S1to S5 in the flowchart shown in FIG. 32 except that the parallaxquantity adjustment at Step S16 is replaced by the parallax quantityadjustment at Step S17. Illustratively, the process from Step S21 toStep S26 shown in FIG. 35 is implemented. The flow of this process willbe described with reference to the flowchart shown in FIG. 35.

At Step S21, the parallax quantity acquisition area in a target3-dimensional image to be processed is initialized with the entireimage.

At Step S22, the amount of protrusion and the amount of depth arecompared every pixel in the parallax quantity acquisition area toacquire the maximum amount of protrusion f′ and the maximum amount ofdepth b′ in the parallax quantity acquisition area. In the presentembodiment, it is assumed that the information on the amount ofprotrusion and the amount of depth for every pixel of the image is givenin advance as tag information. However, there is a method ofautomatically determining the amounts for each pixel by stereo matching,meaning that the method of acquisition is not limited to use of taginformation. Alternatively, it is not necessary to earn the amounts ofprotrusion and depth for all pixels; these values may be extracted fromsome distinctive pixels in the parallax quantity acquisition area.

At Step S23, it is determined whether a stereoscopic view can beobtained by adjusting the parallax quantity by shifting the3-dimensional image as a whole. It is possible to create display of acomfortable stereoscopic view by setting back the entire 3-dimensionalimage from the stereoscopic display with a shift of f-th_f′ as shown inFIG. 34. However, when f′-b′ is greater than th_f-th_b, only a simpleshift of the whole image left or right does not work well for correctionof protrusion.

Accordingly, when f′-b′ is greater than th_f-th_b, the CPU of the PCdetermines that no stereoscopic view can be obtained by adjustment ofthe parallax quantity, and cuts down the parallax quantity acquisitionarea at Step S24, and repeats the process from Step S22. In the presentembodiment, as shown in FIG. 37, the entire 3-dimensional image is setup as an initial range L1 (width: w1 and height: h2). When no correctionprocess can be made with L1, a revised range L2 (width: w2 and height:h2) is set up for correction. When no correction process can be madewith L2, a revised range L3 (width: w3 and height: h3) and so on are setup repeatedly. It is assumed here that wn=0.9×wn−1 and hn=0.9×hn−1, butthe recurrence formula should not be limited to this.

In this way, the CPU of the PC makes the parallax quantity adjustment byshifting the 3-dimensional image at Step S25 when it is determined atStep S23 that a stereoscopic view can be obtained by adjustment of theparallax quantity.

Instead of the center of the image frame, as the area in whichadjustment of stereoscopic effect is made, an object having the greatestprotrusion or the most mark-worthy object is designated beforehand as anobservable point, and adjustment as to stereoscopic effect is madegiving priority to that observable point and its vicinity, whereby it ispossible to create display of a comfortable stereovision.

The Eleventh Embodiment

Any of the embodiments described heretofore is not limited to executionon the applications on PCs, but is executable on TV apparatus, PDAs,cellular phones and others. Application to these will be describedhereinbelow as the eleventh embodiment of the present invention.

The eleventh embodiment is composed of, as shown in FIG. 38, a3-dimensional image data source 10, a temporary data processing storage11, a 3-dimensional image display unit 12, a magnification ratioselector 13, a parallax adjustment unit 14 and a warning decision unit15 and a warning display unit 16.

First, color data and parallax data of 3-dimensional image data forevery pixel from 3-dimensional image data source 10, and the size andresolution of the display device and the magnitude of crosstalk from3-dimensional image display unit 12, are delivered and stored in dataprocessing storage 11. Here, as a 3-dimensional image data source 10, amagnetic disk, semiconductor memory, data transmission via a wired orwireless network, and others can be considered. As a temporary dataprocessing storage, a semiconductor memory, magnetic disk and othersbuilt into PDAs, cellular phones etc., can be considered.

Next, the user selects the magnification ratio for display of a3-dimensional image through magnification ratio selector 13. As themagnification ratio selector 13, buttons, a dial or the like forenlargement and reduction can be considered.

In parallax adjustment unit 14, a similar process to that of Steps S1 toS5 in the eighth embodiment is effected on the 3-dimensional image datastored in data processing storage 11. Specifically, the displayinformation stored in data processing storage 11 is acquired so as toobtain the parallax range in which stereoscopic view of 3-dimensionalimage data can be obtained, then the parallax quantity of the3-dimensional image is compared to the parallax range, whereby it isdecided whether the parallax quantity of the 3-dimensional image fallswithin the parallax range for permitting stereovision.

If it falls out of the parallax range for permitting stereoscopic view,warning processor 15 gives a warning on 3-dimensional image display unit12. The wording for display of warning is stored in data processingstorage 11, and warning processor 15 makes a judgment based on theextent the parallax quantity of the 3-dimensional image deviates fromthe parallax range for permitting the stereoscopic view. In this way,the wording for warning is once stored in data processing storage 11 andis overlay displayed on 3-dimensional image display unit 12 to give awarning.

Warning may be given using a separate dedicated warning display unitother than 3-dimensional image display unit 12 or may be given withwarning sound or the like by using a speaker or headphone.

INDUSTRIAL APPLICABILITY

The present invention provides a 3-dimensional image creating apparatusand a 3-dimensional image processing apparatus for displaying3-dimensional images, and enables the user to confirm the content of3-dimensional images in a proper manner and is suitable for an apparatusthat presents the content of a 3-dimensional image to the user even ifit is enlarged or reduced.

1. A 3-dimensional image processing apparatus, comprising: a parallaxquantity acquisition means for acquiring a parallax quantity of a3-dimensional image; a decision means for deciding whether the parallaxquantity of the 3-dimensional image falls within a parallax rangesuitable for a stereoscopic view; and a warning means for warning auser, wherein the warning means warns the user when the decision meansdetermines that the parallax quantity falls out of the parallax range.2. The 3-dimensional image processing apparatus according to claim 1,further comprising a magnification ratio selecting means for displayingan enlarged or reduced view of the 3-dimensional image, wherein thedecision means determines whether the parallax quantity of the3-dimensional image that has been enlarged or reduced based on aselected magnification ratio falls within the parallax range.
 3. The3-dimensional image processing apparatus according to claim 1, whereinthe decision means makes a deciding process based on a partial area ofthe 3- dimensional image.
 4. The 3-dimensional image processingapparatus according to claim 1, wherein the parallax quantityacquisition means uses a resolution of a stereoscopic display fordisplaying the 3-dimensional image, a size of a stereoscopic display fordisplaying the 3-dimensional image, or a resolution and size of astereoscopic display for displaying the 3-dimensional image.
 5. The3-dimensional image processing apparatus according to claim 1, furthercomprising a parallax range acquisition means for determining andacquiring the parallax range suitable for the stereoscopic view, whereinthe parallax range acquisition means determines and acquires theparallax range by using a capability of separating left and right imagesof a stereoscopic display for displaying the 3-dimensional image.
 6. The3-dimensional image processing apparatus according to claim 1, whereinthe parallax quantity acquisition means uses data previously tagged tothe 3-dimensional image.
 7. A 3-dimensional image processing apparatus,comprising: a parallax quantity acquisition means for acquiring aparallax quantity of a 3-dimensional image; a decision means fordeciding whether the parallax quantity of the 3-dimensional image fallswithin a parallax range suitable for a stereoscopic view; a parallaxadjustment means for adjusting the parallax quantity of the3-dimensional image; and a designating means for designating anobservable point for adjustment of the parallax quantity, wherein theparallax adjustment means adjusts the parallax quantity giving priorityto an area around the observable point when the decision meansdetermines that the parallax quantity falls out of the parallax range.8. The 3-dimensional image processing apparatus according to claim 7,further comprising a magnification ratio selecting means for displayingan enlarged or reduced view of the 3-dimensional image, wherein thedecision means determines whether the parallax quantity of the3-dimensional image that has been enlarged or reduced based on aselected magnification ratio falls within the parallax range.
 9. The3-dimensional image processing apparatus according to claim 7, whereinthe decision means makes a deciding process based on a partial area ofthe 3-dimensional image.
 10. The 3-dimensional image processingapparatus according to claim 7, wherein the parallax quantityacquisition means uses a resolution of a stereoscopic display fordisplaying the 3-dimensional image, a size of a stereoscopic display fordisplaying the 3-dimensional image, or a resolution and size of astereoscopic display for displaying the 3-dimensional image.
 11. The3-dimensional image processing apparatus according to claim 7, furthercomprising a parallax range acquisition means for determining andacquiring the parallax range suitable for the stereoscopic view, whereinthe parallax range acquisition means determines and acquires theparallax range by using a capability of separating left and right imagesof a stereoscopic display for displaying the 3-dimensional image. 12.The 3-dimensional image processing apparatus according to claim 7,wherein the parallax quantity acquisition means uses data previouslytagged to the 3-dimensional image.
 13. A 3-dimensional image processingapparatus, comprising: a parallax quantity acquisition unit acquiring aparallax quantity of a 3-dimensional image; a decision unit decidingwhether the parallax quantity of the 3-dimensional image falls within aparallax range suitable for a stereoscopic view; and a warning unitwarning a user, wherein the warning unit warns the user when thedecision unit determines that the parallax quantity falls out of theparallax range.
 14. A 3-dimensional image processing apparatus,comprising: a parallax quantity acquisition unit acquiring a parallaxquantity of a 3-dimensional image; a decision unit deciding whether theparallax quantity of the 3-dimensional image falls within a parallaxrange suitable for a stereoscopic view; a parallax adjustment unitadjusting the parallax quantity of the 3-dimensional image; and adesignating unit designating an observable point for adjustment of theparallax quantity, wherein the parallax adjustment unit adjusts theparallax quantity giving priority to an area around the observable pointwhen the decision unit determines that the parallax quantity falls outof the parallax range.
 15. The 3-dimensional image processing apparatusaccording to claim 13, further comprising a magnification ratioselecting unit displaying an enlarged or reduced view of the3-dimensional image, wherein the decision unit determines whether theparallax quantity of the 3-dimensional image that has been enlarged orreduced based on a selected magnification ratio falls within theparallax range.
 16. The 3-dimensional image processing apparatusaccording to claim 13, wherein the decision unit makes a decidingprocess based on a partial area of the 3-dimensional image.
 17. The3-dimensional image processing apparatus according to claim 13, whereinthe parallax quantity acquisition unit uses a resolution of astereoscopic display for displaying the 3-dimensional image, a size of astereoscopic display for displaying the 3-dimensional image, or aresolution and size of a stereoscopic display for displaying the3-dimensional image.
 18. The 3-dimensional image processing apparatusaccording to claim 13, further comprising a parallax range acquisitionunit determining and acquiring the parallax range suitable for thestereoscopic view, wherein the parallax range acquisition unitdetermines and acquires the parallax range by using a capability ofseparating left and right images of a stereoscopic display fordisplaying the 3-dimensional image.
 19. The 3-dimensional imageprocessing apparatus according to claim 13, wherein the parallaxquantity acquisition unit uses data previously tagged to the3-dimensional image.
 20. The 3-dimensional image processing apparatusaccording to claim 14, further comprising a magnification ratioselecting unit displaying an enlarged or reduced view of the3-dimensional image, wherein the decision unit determines whether theparallax quantity of the 3-dimensional image that has been enlarged orreduced based on a selected magnification ratio falls within theparallax range.
 21. The 3-dimensional image processing apparatusaccording to claim 14, wherein the decision unit makes a decidingprocess based on a partial area of the 3-dimensional image.
 22. The3-dimensional image processing apparatus according to claim 14, whereinthe parallax quantity acquisition unit uses a resolution of astereoscopic display for displaying the 3-dimensional image, a size of astereoscopic display for displaying the 3-dimensional image, or aresolution and size of a stereoscopic display for displaying the3-dimensional image.
 23. The 3-dimensional image processing apparatusaccording to claim 14, further comprising a parallax range acquisitionunit determining and acquiring the parallax range suitable for thestereoscopic view, wherein the parallax range acquisition unitdetermines and acquires the parallax range by using a capability ofseparating left and right images of a stereoscopic display fordisplaying the 3-dimensional image.
 24. The 3-dimensional imageprocessing apparatus according to claim 14, wherein the parallaxquantity acquisition unit uses data previously tagged to the3-dimensional image.